Image capture apparatus and method for controlling the same

ABSTRACT

Disclosed is an image capture apparatus that has a subject tracking functionality of sequentially detecting a predetermined subject in images that are continuously shot, and a method for controlling the same. During tracking of the subject, degrees of reliability of tracking are determined. When the detected face is set as a template image, the template image is not changed until it is determined a predetermined plurality of times in succession that the degrees of reliability of tracking are low. It is thus possible to achieve a subject tracking functionality that is hardly affected by false detections.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image capture apparatus and a methodfor controlling the same, in particular, to an image capture apparatusthat has a functionality to track a subject, and a method forcontrolling the same.

2. Description of the Related Art

Conventionally, in cameras with a solid-state image sensor, afunctionality to detect a given subject from a frame of a moving image,such as a live view image, and to track the subject in a subsequentframe has been put to practical use. Cameras having such a subjecttracking functionality can detect the location of the subject in theimage so as to appropriately perform an automatic focus adjustment andan automatic exposure control on the subject. For example, a camera hasbeen commercialized that detects a human face and sets it as a trackingtarget, thereby continuously focusing on the detected face. Further,there are also cameras that reset a tracking target, if they have lost asubject during tracking of the subject, and start the tracking again.

For example, Patent Document 1 (Japanese Patent Laid-Open No.2011-27847) discloses a camera that, if it has detected human faces inan image, automatically selects the largest face of the detected humanfaces as a tracking target.

Further, Patent Document 2 (Japanese Patent Laid-Open No. 2009-124565)discloses a camera that performs a face detection operation, then sets atracking target, and starts a tracking operation, the camera having afunctionality to perform a face detection operation again, and to reseta tracking target, if it has lost the tracking target.

Both Patent Documents 1 and 2 teach that, if no face is detected, thepreviously-set subject in an automatic focus detection area is set asthe tracking target.

However, in the conventional technologies disclosed in Patent Documents1 and 2, no countermeasure against false face detection is proposed, sothat a falsely detected area may be taken as a face detection area to betracked.

SUMMARY OF THE INVENTION

The present invention has been accomplished in view of such a problem ofthe conventional technologies. The present invention provides a subjecttracking function that is based on subject detection and having improvedtolerance or robustness to false detection of a subject.

According to an aspect of the present invention, there is provided animage capture apparatus comprising: a face detection unit configured todetect a face that is included in an image; a subject detecting unitconfigured to detect, from an image obtained by shooting, a area that issimilar to a template image; and a control unit configured to achieve afunctionality of tracking a subject by making the subject detecting unitsequentially detect a area similar to a template image of the subjectarea in images that are continuously shot; wherein the control unit isfurther configured to determine, during tracking of the subject,reliability of the detection by the subject detecting unit and, when theface detected by the face detection unit is set as the template image,not to change the template image until it is determined a predeterminedplurality of times in succession that reliability of the detection bythe subject detecting unit is low.

According to another aspect of the present invention, there is provideda method for controlling an image capture apparatus, the image captureapparatus comprising: a face detection unit configured to detect a facethat is included in an image; and a subject detecting unit configured todetect, from an image obtained by shooting, a area that is similar to atemplate image, the method comprising: a controlling step of achieving afunctionality of tracking a subject by making the subject detecting unitsequentially detect a area similar to a template image of the subjectarea in images that are continuously shot, in the controlling step,during tracking of the subject, reliability of the detection by thesubject detecting unit being determined, and when the face detected bythe face detection unit is set as the template image, the template imagebeing not changed until it is determined a predetermined plurality oftimes in succession that reliability of the detection by the subjectdetecting unit is low.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an example of control of a trackingmode shift based on face detection results and reliability of trackingin an image capture apparatus according to an embodiment of the presentinvention.

FIG. 2 is a sectional side view illustrating a digital single-lensreflex camera as an example of the image capture apparatus according tothe embodiment of the present invention.

FIG. 3 is a block diagram illustrating an example of a configuration ofa functionality of the digital single-lens reflex camera according tothe embodiment of the present invention.

FIG. 4 is a diagram illustrating processing sequence of an AF controlunit, a system control unit, and an AE control unit of FIG. 2, and amutual relationship among them.

FIG. 5 is a diagram illustrating a processing sequence of an AF controlunit, a system control unit, and an AE control unit of FIG. 2, and amutual relationship among them.

FIGS. 6A to 6D are diagrams illustrating examples of optical finderimages of the image capture apparatus according to the embodiment of thepresent invention that is tracking the subject.

FIG. 7 is a flowchart illustrating template image generation processingof the image capture apparatus according to the embodiment of thepresent invention.

FIG. 8 is a diagram illustrating an example of a relation between a facedetection area and a photometric area of the image capture apparatusaccording to the embodiment of the present invention.

DESCRIPTION OF THE EMBODIMENTS

An exemplary embodiment of the present invention will now be describedin detail in accordance with the accompanying drawings.

FIG. 2 is a sectional side view illustrating an example of anarrangement of main constituent components of a digital single-lensreflex camera (DSLR) as an example of an image capture apparatusaccording to an embodiment of the present invention. The DSLR includes amain body 201 and an imaging lens 202 that is attached to the front ofthe main body 201. The imaging lens 202 is exchangeable, and the mainbody 201 and the imaging lens 202 are connected to each othermechanically and electrically via a lens mount (not shown). The imaginglens 202 includes a focusing lens 213 and a diaphragm 214, which areoperated by electrical control of the main body 201 via a group of lensmount contacts 215.

A main mirror 203 is a half mirror, and a sub mirror 204 is providedbehind the main mirror 203. In the illustrated state in which the mirroris not raised, the main mirror 203 reflects part of a bundle of lightfrom the imaging lens 202 so that it is incident on a finder opticalsystem provided above, and the sub mirror 204 reflects transmitted lightof the bundle of light so that it is incident on an AF device 205provided below. In the state in which the mirror is raised, for example,at the time of exposure, both mirrors retract from the optical path.

The AF device 205 is an AF sensor of a phase-difference detection type.By imaging a secondary imaging surface of the imaging lens 202 on afocus detection line sensor included in the AF device 205, an amount ofdefocusing and a focus direction of the imaging lens 202 are detected,and the focusing lens 213 is driven on the basis of the detectionresults, thereby performing an automatic focus adjustment. Sincephase-difference detection type focus control is well-known and notdirectly related to the present invention, descriptions of a detailedconfiguration of the AF device 205 and the focus control are omitted.

A focusing screen 206 is a transmissive type liquid crystal display thatis arranged on an imaging surface of the imaging lens 202 on whichimages are to be formed when the imaging lens 202 serves as a finderoptical system, and displays, in a superimposed manner, informationrelating to a focus area and the like on an optical image having a fieldof view formed by the imaging lens 202. By observing the focusing screen206 from an eye piece 209 via a pentaprism 207 for changing a finderoptical path, a user can confirm the field of view of an image to becaptured, a focus area, and the like.

An AE device 208 changes an optical finder image into photometric imagedata using an AE sensor that has a plurality of pixels, in order toobserve the brightness of the subject. The AE sensor may have the sameconfiguration as that of an image sensor 211, except in the number ofpixels, but it is referred to as an “AE sensor” because it is usedmainly for detecting a brightness of the subject. The photometric imagedata has R, G, and B components. The AE device 208 performs, using thegenerated photometric image data, brightness detection, exposurecontrol, tracking, and face detection of the subject.

The image sensor 211 is a CMOS image sensor, a CCD image sensor, or thelike, in which a plurality of pixels each including a photoelectricconversion element are arranged. By retracting the main mirror 203 andthe sub mirror 204 from the shooting optical path, and by opening acurtain shutter 210, the image sensor 211 is exposed and an opticalimage of the subject is shot.

A display 212 is, for example, a liquid crystal display (LCD), used asan electronic view finder while displaying live view images, and usedfor displaying various types of setting values, shooting information,apparatus information, GUI screens, captured images, and the like.

FIG. 3 is a block diagram illustrating an example of a functionalconfiguration of a DSLR of the present embodiment. Note that the samereference numerals are given to the same constituent elements as thosein FIG. 2.

An operation unit 301 detects operation performed by the user via abutton, a switch, a dial, a connection equipment, and the like that aremounted on the main body 201 and/or the imaging lens 202, and outputs asignal that corresponds to the content of the operation to the systemcontrol unit 306. The operation unit 301 sends signals SW1 and SW2according to a push amount of a release button. Specifically, theoperation unit 301 outputs the signal SW1 to the system control unit 306when the release button is pushed approximately half way down, andoutputs the signal SW2 to the system control unit 306 when the releasebutton is further pushed to the predetermined extent. Note that thestate of the release button being pushed so that the signal SW1 isoutput is referred to as a “half way down push”, and the state of therelease button being pushed so that the signal SW2 is output is referredto as a “all the way down push”. The state in which the “half way downpush” of the release button is held is referred to as “SW1 holdingstate”, and the state in which the “all the way down push” of therelease button is held is referred to as “SW2 holding state”. Also, theoperation unit 301 outputs an SW1 cancellation signal to the systemcontrol unit 306 when a push amount of the release button has fallenbelow the push amount for sending the signal SW1, and outputs an SW2cancellation signal to the system control unit 306 when the push amountof the release button has fallen below the push amount for sending thesignal SW2.

As described above, the AF device 205 performs automatic focus detectionof the phase-difference detection type and drives the focusing lens 213to a focus position (automatic focus adjustment) based on the results ofthe automatic focus detection. The AF device 205 is constituted by an AFcontrol unit 304 and a focusing sensor 305. The focusing sensor 305,which is configured by a line sensor that corresponds to, for example,an arrangement of 61 focus areas as illustrated in FIG. 6A, convertsinto electric signals the light that is incident thereon via the submirror 204, and outputs a pair of image signals for each focus area tothe AF control unit 304. The AF control unit 304 calculates, on thebasis of a phase difference between the pair of image signals that wereoutput from the focusing sensor 305, an amount of defocusing for eachfocus area illustrated in FIG. 6A, and selects one focus area to befocused. Then, the AF control unit 304 outputs to the system controlunit 306 a defocus map that has an amount of defocusing for each focusarea as data, and location information of the selected focus area. Thesystem control unit 306 performs focus adjustment operation on the basisof the location of the selected focus area and the defocus map, andcontrols the location of the focusing lens 213 so that it focuses on theselected focus area.

The AE device 208 performs the automatic exposure control and thesubject tracking. The AE device 208 is constituted by the AE controlunit 302 and the photometric sensor 303. The AE control unit 302performs the automatic exposure control on the basis of the photometricimage data that were sequentially read out from the photometric sensor303, and outputs the result thereof to the system control unit 306. Thesystem control unit 306 controls, on the basis of the result of theautomatic exposure control that was output from the AE control unit 302,the F-number of the diaphragm 214 (aperture size) so as to adjust anamount of light that is to be incident on the main body 201.

Furthermore, in the SW1 holding state and during continuous shooting,the AE control unit 302 sequentially detects the subject using thephotometric image data that were continuously shot by the photometricsensor 303, thereby achieving the subject tracking and outputtinginformation on the location of the detected subject to the systemcontrol unit 306. The system control unit 306 outputs to the AF controlunit 304 the location information of the subject that was output by theAE control unit 302.

The AF control unit 304 calculates an amount of defocusing in a focusarea at the location of the subject or near that location that wasoutput by the system control unit 306, and compares the calculationresults with a predetermined threshold. If the result of the comparisonshows that the amount of defocusing is not greater than the threshold,the AF control unit 304 updates the location data on the selected focusarea that will be output to the system control unit 306 from the focusarea that was selected by the AF control unit 304 on the basis of theamount of defocusing for each focus area to a focus area that is locatedon the subject or near the subject.

When the signal SW2 is output from the operation unit 301, the systemcontrol unit 306 raises the main mirror 203 (and the sub mirror 204),and retracts it (them) from the optical path. Then, the system controlunit 306 drives the curtain shutter 210, and exposes the image sensor211 for a time period according to the result of the automatic exposurecontrol. When the exposure time ends, the system control unit 306 lowersthe main mirror 203 (and the sub mirror 204).

The image sensor 211 converts the light that is incident thereon via theimaging lens 202 during the exposure time into an electric signal foreach pixel so as to generate image data, and outputs the image data tothe system control unit 306. The system control unit 306 subjects theimage data that was output from the image sensor 211 to predeterminedimage processing or the like, and displays the image data on the display212 or writes the image data into the image memory device 307.

Hereinafter, operation of the DSLR of the present embodiment will bedescribed.

FIGS. 4 and 5 are diagrams illustrating an operation sequence of the AFcontrol unit 304, the system control unit 306, and the AE control unit302, specifically FIG. 4 illustrates an operation sequence in the SW1holding state, and FIG. 5 illustrates an operation sequence in the SW2holding state. Note that FIGS. 4 and 5 are intended to illustrate theprocessing sequence and operation timing in each control unit, so thatthe size (height) of a block is not associated with the length ofprocessing time even if the processing is performed by the same controlunit, let alone by different control units.

Step S441 indicates the timing at which the release button is pushedhalf way down and the signal SW1 is output from the signal operationunit 301 to the system control unit 306. The system control unit 306that has received the input of the signal SW1, outputs an AF startsignal D01 to the AF control unit 304, and an AE start signal D02 to theAE control unit 302.

In the SW1 holding state, the AE control unit 302 repeatedly performsthe processing from step S401 to step S406 as a single processing unitwith respect to an image in one moving image frame (still image frame)(hereinafter moving image frame and still image frame being referred toas merely “frame”). Similarly, the AF control unit 304 performs theprocessing from step S421 to step S425 as a single processing unit withrespect to an image in one frame. However, both the AE control unit 302and the AF control unit 304 start one processing unit in response toreception of the start signals (the AF start signal and the AE startsignal) that were output from the system control unit 306. Further,subject tracking processing (S403) and focus area re-selectionprocessing (S424), which are included in the single processing unit, arenot performed in the first frame since the AE control unit 302 and theAF control unit 304 start the processing, but from the second frameonward. This is because no tracking target has yet been decided in thefirst frame.

Now, the processing from step S401 to step S406 that is performed in theAE control unit 302 is explained.

In step S401, when receiving the AE start signal D02 from the systemcontrol unit 306, the AE control unit 302 performs accumulation with thephotometric sensor 303 (shooting by the photometric sensor 303). Thisaccumulation (shooting) operation is started simultaneously with theaccumulation operation of the focusing sensor 305 performed by the AFcontrol unit 304 that has received the AF start signal D01 from thesystem control unit 306. Accordingly, the image signals that areobtained by the focusing sensor 305 and the photometric sensor 303 aresignals that are shot at the same timing.

In step S402, the AE control unit 302 reads out the photometric imagedata from the photometric sensor 303, and generates detection targetimage data having RGB color information.

In step S403, the AE control unit 302 performs the subject trackingprocessing for detecting the location of the tracking target(coordinates of the subject) in the detection target image.Specifically, the AE control unit 302 takes a template image that wasgenerated in the previously performed tracking template generationprocessing (S406) as a tracking target image, and performs templatematching on the detection target image that was generated in step S402.“Template matching” refers to a method for detecting an image area thatis included in the detection target image and is identical or similar tothe template image, the detection being performed by evaluating thecorrelation therebetween on the basis of brightness information andcolor information of the images.

Further, the AE control unit 302 calculates degrees of reliability oftracking on the basis of how much the template image, and an image inthe area detected as the tracking target that is included in thedetection target image are correlated. The higher they are correlated,the higher the degree of reliability that is calculated. Then, the AEcontrol unit 302 outputs the subject tracking processing results, asdata D03, which includes the location of the tracking target and thedegree of reliability of tracking, to the system control unit 306. Thisdata is transferred, as data D04, to the AF control unit 304 by thesystem control unit 306.

In step S404, the AE control unit 302 performs exposure control based onthe detection target image that was generated in step S402, and outputsthe exposure control results, as exposure control data D06, to thesystem control unit 306. At that time, if a face was detected in theface detection processing (S405) performed with respect to the image inone frame before, the AE control unit 302 corrects the exposure controlresults so that the face can be shot with correct exposure, on the basisof the brightness information of the detected face area in the samedetection target image of one frame before. In the present embodiment,up to five focus areas that are close to the selected focus area areselected in the detected face and a brightness value (BV value) of thesubject for deciding a shooting condition is calculated from the averagevalue of brightnesses in the face area.

In step S405, the AE control unit 302 subjects the image data that wasread out in step S402 to face detection processing, and outputs adetected face location (representative coordinates in the face area). Ifa plurality of faces were detected, the face that is located closest tothe selected focus area is selected on the basis of data on the selectedfocus area D08 that will be described later and on the detectedlocations of the face, and the face is taken as a candidate for atracking target.

In step S406, the AE control unit 302 generates or updates the trackingtemplate on the basis of the data on the selected focus area D08 thatwill be described later and on the results of the face detectionprocessing in step S405. The detail of the processing will be explainedlater.

The foregoing is the processing of the AE control unit 302. However, asdescribed above, no subject tracking processing in step S403 isperformed if no tracking template was generated (for example, in thefirst frame).

The processing from step S421 to step S425 that is performed by the AFcontrol unit 304 is explained in the following.

In step S421, when receiving the AF start signal D01 that was outputfrom the system control unit 306, the AF control unit 304 starts thecharge accumulation (shooting) using the focusing sensor 305.

In step S422, the AF control unit 304 reads out the image signal fromthe focusing sensor 305.

In step S423, the AF control unit 304 performs the focus detectionoperation based on the image signal that was read out in step S422, andcalculates a focus condition (an amount of defocusing) in the focusarea. Note that, in the first frame after the output of the signal SW1,focus detection is performed in the focus area selected in advance bythe user settings or the like, but in the second frame or later, focusdetection is performed in the focus area selected in the one framebefore and the focus areas close to that frame.

In step S424, the AF control unit 304 receives from the system controlunit 306 the data D04, which is the result of the subject detectionprocessing performed by the AE control unit in step S403 and relates tothe location of the tracking target and degree of reliability oftracking. Then, on the basis of the location of the tracking target andthe degree of reliability of tracking, the AF control unit 304re-selects a focus area. Namely, the AF control unit 304 calculatesamounts of defocusing in five focus areas in total among the selectablefocus areas, the five focus areas including a focus area that is closestto the location of the tracking target (the detected location of thesubject), and other focus areas in upward, downward, leftward, andrightward surrounding focus area. If the calculated minimum amount ofdefocusing is not greater than a predetermined value, the AF controlunit 304 takes the focus area having the calculated minimum amount ofdefocusing as a newly selected focus area, instead of the focus areaselected in step S423.

On the other hand, if all the amounts of defocusing in the five focusareas are greater than the predetermined value, the AF control unit 304does not change the focus area selected in step S423. In this way, bytaking as candidates for re-selection not only the focus area that isclosest to the location of the tracking target but also the surroundingfocus areas, it is possible to perform focus detection of a mainsubject, even if the contrast of the focus area closest to the locationof the tracking target thereto is so low that an amount of defocusingcannot be calculated.

If the selected focus area is decided, the AF control unit 304 outputs,to the system control unit 306, data D05, which includes information onthe amount of defocusing corresponding to the location of the selectedfocus area. With this measure, focus adjustment by the system controlunit starts (S442).

In step S425, the AF control unit 304 calculates focus conditions(amounts of defocusing) for all focus areas that are selectable, so asto update the defocus map, and outputs the updated defocus map, as dataD07, to the system control unit 306.

The system control unit 306 outputs, as data D08, the location of theselected focus area that was received as the data D05, and the defocusmap that was received as the data D07 to the AE control unit 302. Notethat “location of the selected focus area” may refer to informationindicating the specific location of the focus area, but information thatcan specify the focus area, such as focus area number, is alsoapplicable when information on the location of each focus area isseparately available.

Since the calculation of amounts of defocusing for all focus areasrequires time, the response may deteriorate if the focus adjustmentprocessing performed by the system control unit 306 in step S442 isperformed after the defocus map is updated. Therefore, deterioration inresponse of the focus adjustment is prevented by first outputting, tothe system control unit 306 in step S423, information needed foradjusting focus on the selected focus area so that the focus adjustmentprocessing is performed, and then newly supplying a defocus map to thesystem control unit 306.

The foregoing is the processing of the AF control unit 304. Since,however in the first frame, the AE control unit 302 skips the subjecttracking processing in step S403 and does not output the data D04relating to the location of the tracking target, the AF control unit 304skips the focus area re-selection processing in step S424. Here, the AFcontrol unit 304 sets the focus area that was selected in the focusdetection operation and the focus area selecting processing in step S423as the selected focus area, and outputs it, as the data D05, to thesystem control unit 306.

Lastly, the processing of the system control unit 306 is explained.

In step S442, the system control unit 306 drives the focusing lens 213on the basis of the information relating to the location of the selectedfocus area and the amount of defocusing that are received from the AFcontrol unit 304 as the data D05, and performs the focus adjustment suchthat the focusing lens 213 focuses on the selected focus area.

Further, the system control unit 306 synchronizes timings to start theoperations of the AE control unit 302 and the AF control unit 304, andtransfers data between the AE control unit 302 and the AF control unit304.

The details of this operation of the system control unit 306 are asfollows.

In order to synchronize the accumulation operations of the photometricsensor 303 and the focusing sensor 305, the system control unit 306outputs the AF start signal D01 and the AE start signal D02 for theprocessing simultaneously.

The system control unit 306 receives from the AE control unit 302 thedata D03, which relates to the location of the tracking target and thedegree of reliability of tracking, and outputs it, as the data D04,which relates to the location of the tracking target, to the AF controlunit 304 at a predetermined timing.

The system control unit 306 receives, from the AF control unit 304, thedata D05, which relates to the information on the location of theselected focus area, and the data D07, which relates to the defocus mapsof all the focus areas, and outputs them, as the data D08, at thepredetermined timing to the AE control unit 302.

FIG. 5 is a diagram illustrating processing sequence of the AF controlunit 304, the system control unit 306, and the AE control unit 302 whenperforming continuous shooting (sequential shooting) after the releasebutton is pushed all the way down and the signal SW2 is output.

Similar to FIG. 4, FIG. 5 is also intended to illustrate the processingsequence and the operation timing in each control unit, so that the size(height) of a block is not associated with the length of processing timeeven if the processing are performed by the same control unit, let aloneby different control units. Further, in FIG. 5, the processing to whichthe same numbers are added as those in FIG. 4 are the same as theprocessing explained in FIG. 4, and duplicate explanations are omitted.

In step S501, when detecting the output of the signal SW2 from theoperation unit 301, the system control unit 306 starts a series ofoperations of a mirror raising operation (S502), a main shootingoperation (S503), and a mirror lowering operation (S504). Each operationis explained in detail later.

Further, the system control unit 306 outputs a release start signal D09to the AF control unit 304 and a release start signal D10 to the AEcontrol unit 302. The AF control unit 304 and the AE control unit 302that have received the release start signals D09 and D10 interrupt theprocessing that they are carrying out at that time. After completing themirror lowering operation in step S504, the system control unit outputsthe AF start signal D01 to the AF control unit 304, and the AE startsignal D02 to the AE control unit 302.

The AF control unit 304 that has received the AF start signal D01performs the processing from step S421 to step S425, and the AE controlunit 302 that received the AE start signal D02 performs the processingfrom step S401 to step S406. During the SW2 holding state, the systemcontrol unit 306 outputs the AF start signal D01 to the AF control unit304 and the AE start signal D02 to the AE control unit 302 at a timingat which the mirror lowering operation in step S504 for each frame iscompleted.

In the following, the processing from steps S502 to S504 is explained indetail.

In step S502, the system control unit 306 raises the main mirror 203 andthe sub mirror 204, so that they retract from the optical path. Also,the system control unit 306 drives the diaphragm 214 on the basis of theexposure control result received as the data D06 directly from the AEcontrol unit 302, and controls an aperture of the diaphragm.

In step S503, the system control unit 306 drives the curtain shutter 210on the basis of the exposure control results, and exposes the imagesensor 211 for a time period according to the exposure control results.

In step S504, the system control unit 306 lowers the main mirror 203 andthe sub mirror 204.

The image sensor 211 accumulates electric charge that is generated bylight incident on each pixel via the imaging lens 202 during theexposure time, and converts the electric charge into electric signals soas to generate image data, outputting them to the system control unit306. The system control unit 306 subjects the image data that was outputfrom the image sensor 211 to predetermined image processing and thelike, and the image data is displayed on the display 212 or written intothe image memory device 307.

As long as the SW2 state is maintained, the system control unit 306repeats the processing from step S442 to step S504 as one processingunit, thus achieving continuous shooting.

Next, the operation sequence of the tracking template generationprocessing performed by the AE control unit 302 in step S406 isexplained with reference to the flowchart illustrated in FIG. 7.

In step S701, the AE control unit 302 sets a face tracking startdetermination flag to “False”. This flag is used in step S708 fordetermination of whether or not to update the template image.

In step S702, the AE control unit 302 determines whether or not thecurrent frame is the first frame since starting to output the signalSW1. If it is the first frame, the template image was not generated andthus the AE control unit 302 extracts, in step S711, the selected focusarea of the photometric image as a template image, and ends the trackingtemplate generation processing.

On the other hand, if it is determined in step S702 that the currentframe is the second frame or later, then the AE control unit 302determines, in step S703, whether or not the current template image isgenerated on the basis of the face detection area. If the template imageis not generated based on the face detection area, then the AE controlunit 302 sets, in step S707, the face tracking start determination flagto “True”, and advances the processing to step S708. If the templateimage is generated based on the face detection area, then the AE controlunit 302 advances the processing to step S704.

This measure is to prevent updating of a template image even if thetemplate image is generated based on the face detection area (that is,when the template image is a face) and a new face is detected during thetracking processing. This prevents the face of the detection target frombeing changed on the way.

In step S704, the AE control unit 302 determines whether or not thelapsed time since the last template image was generated is longer thanor equal to a predetermined time period. Although, here, the lapsed timeis determined according to the number of frames that were shot, it isalso possible to measure the lapsed time using a timer or the like. TheAE control unit 302 advances the processing to step S705 if the lapsedtime is longer than or equal to the predetermined time period, and tostep S708 if the lapsed time is shorter than the predetermined timeperiod. In the SW1 holding state, the number of “frames” of the movingimage is used because a live view image (moving image) is being shot(the same applies to shooting of a moving image for recording), whereasin the SW2 holding state, the number of “frames” of still images thatare being continuously shot is used. Ordinarily, if the frame rate whenshooting a moving image and the shooting interval when performingcontinuous shooting differ from each other, the lengths of lapsed timealso differ from each other although the number of frames of a movingimage and the number of frames of still images are the same. Therefore,it is possible to convert the lapsed time to actual time if needed. Inany case, the determination here intends to determine whether or not thetemplate image should be updated on the basis of the lapsed time, and isnot limited to a specific method. For convenience, “frame of a movingimage” and “frame of still images” are hereinafter collectively referredto as “frame”.

In step S705, the AE control unit 302 determines whether or not a degreeof reliability of tracking in a predetermined number of frames that areclose to each other is the predetermined value at most. If the degree ofreliability of tracking in all the predetermined number of the framesthat are close to each other is the predetermined value at most, the AEcontrol unit 302 determines that it has lost the tracking target andsets, in step S706, the face tracking start determination flag to“True”, and advances the processing to step S708. If there is any framewhose degree of reliability of tracking exceeds the predetermined value,the AE control unit 302 does nothing and advances the processing to stepS708.

In step S708, the AE control unit 302 advances the processing to stepS709 if the face tracking start determination flag that was set in stepsS701, S706, and S707 is “True”, and ends the tracking templategeneration processing if it is “False”.

In step S709, the AE control unit 302 determines whether or not, in theface detection processing S405, a face was detected in the current frameand the one frame before in succession. If faces were detected insuccession, the AE control unit 302 sets (updates), in step S710, animage in the face detection area of the photometric image as thetemplate image, and ends the tracking template generation processingS406. If no face is detected in any frame, the AE control unit 302 doesnot set (update) a template image and ends the tracking templategeneration processing in step S406.

The foregoing is the explanation of the processing performed in thetracking template generation processing of step S406.

With the processing in step S705, the template image is not updated atthe location of face detection until a state of a low degree ofreliability of tracking continues for a predetermined number of frames,so that it is possible to prevent the tracking target from being changedto another face of a subject, such as a passer-by in the background,other than the main subject. Also, even if the main subject is notdetected temporarily because it is hidden behind an obstacle or changesits direction, the template image is kept to be held for thepredetermined number of frames, so that it is possible to start againtracking when the main subject can be detected again.

On the other hand, if the state of a low degree of reliability oftracking continues for a predetermined number of frames, the AE controlunit 302 determines that it has lost the tracking target and immediatelysets, with the operation in step S709, a newly detected face to thetemplate image as a new tracking target, so that it is possible toautomatically change the tracking target.

Hereinafter, the relation between results of the subject trackingprocessing and the operation of the tracking template generationprocessing of the present embodiment is explained further in detail withreference to FIGS. 1, 6A to 6D, and 8.

FIG. 1 is diagram illustrating an example of control of tracking modeshift based on face detection results and reliability of tracking, theface detection and the tracking being performed in the SW1 holding stateby the AE control unit 302. In the present embodiment, the subjecttracking processing and the tracking template generation processing havethree tracking modes that depend on a subject as a tracking target.Namely, the three tracking modes are a face tracking mode 106, an AFtracking mode 107, and a tracking OFF mode 108.

The face tracking mode 106 is a mode in which tracking is performed withan image area centered on a area where a face is detected being set as atemplate image. The AF tracking mode 107 is a mode in which tracking isperformed with an image area centered on a focus area selected by the AFbeing set as a template image. The tracking OFF mode 108 is a mode inwhich no tracking is performed.

In FIG. 1, a release operation 101 shows a state of the release button.Here, “SW1 ON” denotes a timing at which output of the signal SW1 isdetected, the signal SW1 being output by the release button being pushedhalf way down.

The number of frames of AE 102 shows the number of frames that haveelapsed since the AE control unit 302 has first received the AE startsignal D02 and started the operation.

Degree of reliability of tracking 103 schematically shows thereliability of the subject tracking results that were calculated in thesubject tracking processing in step S403. Here, ∘ denotes the case wherea degree of reliability is determined to be higher than a predeterminedthreshold, whereas x denotes the case where a degree of reliability isdetermined to be lower than the predetermined threshold. A method forcalculating the degree of reliability of tracking 103 will be explainedlater.

The face detection result 104 shows whether or not at least one face wasdetected in the face detection processing in step S405. Here, ∘ denotesthe case that at least one face was detected, whereas x denotes the casethat no face was detected.

Step S110 to Step S115 show frames that relate to mode shifts fortracking the tracking target.

Step S110 is the first frame since the AE control unit 302 started theoperation. At this point of time, the tracking template generationprocessing has not been yet performed, so that no template image fortracking has been generated. Therefore, the tracking OFF mode 108 ismaintained, and no tracking processing is performed.

Step S111 shows the state in the next frame (frame number 2). Thesubject tracking starts by using a template image that was generated inthe tracking template generation processing (S406) from a photometricimage generated in a sensor read-out processing (S402) in the FIG. 4 inthe first frame. In this frame, no face is detected, so that the modeshifts to the AF tracking mode 107. In this case, the tracking target isa subject that is present at the location of the selected focus area. Inthis way, it is possible to track any subject even if no face wasdetected.

An example of an optical finder image in the AF tracking mode using thetemplate image that was generated in the frame of step S110 isschematically illustrated in FIG. 6A. As described above, rectanglesillustrated in FIG. 6A are frames (focus area frames) that each show afocus area that can be set, in which the rectangles with dotted linesare not displayed.

A user selected focus area 611 with a thick solid line as illustrated inFIG. 6A is a focus area that is being selected by the user at that timewhen the release button is pushed half way down. When receiving the AFstart signal D01 in FIG. 4, which was output by the release button beingpushed half way down, the AF control unit 304 sets the user selectedfocus area 611 as a selected focus area in the focus area selectingprocessing (S423) in FIG. 4.

A focus area 612 represented with a solid line that is shown below theuser selected focus area 611 shows, among the focus areas locatedupward, downward, leftward, and rightward the selected focus area, afocus area in which a difference in amounts of defocusing between thefocus area and the selected focus area is not greater than apredetermined threshold.

Since focus area frames with solid lines are displayed on the focusingscreen 206 illustrated in FIG. 2, the user can view, through the finder,the selected focus area, and surrounding focus areas that each have alittle difference in the amount of defocusing between the focus area andthe selected focus area. The focus area frames with dotted lines are notdisplayed on the focusing screen 206.

The tracking template generation processing S406 in FIG. 4 startstracking from step S111 with the location of the user selected focusarea 611 being set as a tracking target.

Step S112 shows the state in the fourth frame. The face detection result104 sequentially shows ∘ in the third and fourth frames, that is, a faceis detected twice in succession. In such a case, the AE control unit 302updates, in the tracking template generation processing (S406) in FIG.4, the template image to an image that is cut out from the facedetection area.

The reason why only a face that was detected twice in succession is setas a tracking target is to reduce the probability of setting a falselydetected area as a tracking target in the case of false detection in theface detection processing (S405) in FIG. 4. Since the detection rate ofa real face is higher than in the case where an object is falselydetected as a face, the more often sequential detection is performed,the lower the probability of setting the falsely detected object as atracking target is. However, if the conditions for the number of thesequential detection are increased, then it will take longer time untila real face is set as a tracking target, and therefore the number of thesequential detection is here experientially set to two.

The area where the face is detected is taken as a new tracking target,so that the mode shifts to the face tracking mode 106.

This makes it possible, if a face is detected in the SW1 holding stateand during continuous shooting (in the SW2 holding state), toautomatically focus on faces without the need of the user manuallyre-setting the focus area.

Change of the optical finder image when tracking the target in step S112is schematically illustrated in FIGS. 6B and 6C. In the example of FIG.1, since no face is detected up to the second frame, tracking isperformed using the template image 601 that was set in step S110 (thefirst frame) as with in FIG. 6B, and a body unit of the person is keptto be selected as the focus area.

In step S112 in the fourth frame, since the face that was sequentiallydetected in the third and fourth frames is set as the template image,the face is now tracked using a new template image 602 as in FIG. 6C.Consequently, the focus area shifts to the face, making it possible toshoot an image that is focused on the face.

Step S113 shows the sixth frame, in which ∘ is indicated as the facedetection result 104, which was x in the fifth frame. However, thedetected face is not necessarily of the same person whose face wasdetected in step S112. If the template image is updated each time a newface is detected, there is a risk that the person set as a trackingtarget is often changed.

Particularly when face detection is performed in an image that was shotby a photometric sensor, the image obtained by the photometric sensorincludes an indication of focus area frames, so that the face of theperson who is being tracked cannot be detected during tracking due tothe indication of the focus area frames (see FIG. 6D), but only a faceof another person, instead of the person of the tracking target, may bedetected.

This is because the AE device 208 detects faces on the basis of imagesin a field of view that is projected or displayed on the focusing screen206, which is a transmissive type display device (see FIG. 2). That is,since the focus area frames are displayed on the focusing screen 206,which is the transmissive type display device, part of the main subjectthat is a tracking target may be hidden behind the focus area frames anddifficult to be viewed, in the image observed by the AE device 208.

The focus area frames that are displayed on the focusing screen 206 arethe selected focus areas in one frame before, and the face that is atracking target is more likely to be present near the selected focusareas in the one frame before. Therefore, in the face detectionprocessing by the AE device 208, the face that is a tracking target ismore likely to be affected by the focus area frames, and the face thatis not a tracking target is less likely to be affected by the focus areaframes, as compared with the face that is a tracking target. Therefore,if face detection is performed by the AE device 208 that observes afield of view including the indication of focus area frames, a state islikely to be caused where the face of the tracking target cannot bedetected during the face tracking because it is hidden behind theindication of the focus area frames, and only another face is detected.

Further, independently from whether or not the selected focus area frameis displayed, there is a possibility of false detection in which part ofa person other than his face may be mistaken for a face. If a subject tobe tracked is mistaken, another subject differing from the subjectintended by the user is continuously focused.

Therefore, in the present embodiment, no template image is updated inthe face tracking mode 106. Specifically, the tracking templategeneration processing (S406) is performed without reference to theresults of the face detection processing (S405). This makes it possibleto continuously track the face of the person that was first set as atemplate image.

On the other hand, in order to effectively use the face detectionfunctionality, in the exposure control processing (S404) in step FIG. 4,up to five focus areas that are close to the selected focus area areselected in the detected face and, from the average value thereof, abrightness value (BV value) of the subject for use for deciding ashooting condition is calculated. By deciding a shooting condition onthe basis of this BV value, it is possible to set appropriate exposuresfor a plurality of faces respectively.

For example, shooting as illustrated in FIG. 8 is considered in whichtwo persons are main subjects, and a background is a scene of a brightsky. In such a scene, if exposure control is performed on the basis ofphotometric information (a brightness value) in the entire photometricarea 1201, a BV value that is reflected by the bright background may beobtained and thus a shooting condition that lacks exposure to the mainsubjects may be decided. In such a circumstance, by obtaining a BV valueon the basis of the face detection area 1202, it is possible to decide ashooting condition in which the faces of the main subjects are shot withthe correct exposure. In addition to the configuration in which ashooting condition is decided depending only on the BV value based onthe face detection area, a configuration is also possible such that theBV value based on the photometric area 1201 is corrected so as to have alittle difference from the BV value based on the face detection area1202.

Accordingly, since the information on the face area for the exposurecontrol is used, the face detection processing is performed for eachframe even when the tracking template generation processing in FIG. 4 isperformed without reference to the face detection results.

An aspect in which the face detected in step S112 is continuouslytracked is shown in FIG. 6D. In step S113, tracking is performed usingthe template image 602 that was generated in step S112.

Step S114 shows the state in the thirteenth frame. In step S114, degreesof reliability of tracking are low (x) in three successive frames fromthe eleventh frame. If a state like that in which degrees of reliabilityof tracking are lower than a predetermined threshold continues for apredetermined number of successive frames (here, three frames), the AEcontrol unit 302 determines that it has lost the tracking target in thesubject tracking processing (S403), and shifts the mode to the trackingOFF mode 108. However, if a face was detected in the current frame (thethirteenth frame) and one frame before (the twelfth frame) (twice insuccession) as illustrated in FIG. 1, the AE control unit 302 sets thedetected face as a new tracking target. Then, the mode again shifts tothe face tracking mode 106 (S115).

As described above, when reliability of tracking is low in apredetermined number of successive frames (a plurality of times insuccession) and it is determined that the tracking target is lost, if aface is detected in a predetermined number of successive processingframes including the current frame, then the detection target is changedto the currently detected face. Accordingly, without shifting to thetracking OFF mode 108, the face tracking mode substantially continueswhile the tracking target is changed.

On the other hand, when it is determined that the tracking target islost and a face was not detected in a predetermined number of successiveframes (a plurality of times in succession), even if a face is detectedin the current frame, the detected face is not set as a new trackingtarget. Therefore, a possibility of making a falsely detected person ora person who entered the field of view by accident be set as a trackingtarget is suppressed, making it possible to perform subject trackingthat is robust against factors that may bring the tracking operationinto failure, such as false detection of a face or appearance of anunintended person.

While no face is detected and the AE control unit 302 does not performsubject tracking, the AF control unit 304 decides a main subject (afocus area to be selected) according to the results of the focusdetection operation and of the focus area selecting processing (S423).When the state of the low degree of reliability of tracking continuesfor a predetermined number of frames, it is likely that the trackingtarget intended by the user cannot be tracked correctly. Since this mayadversely affect on the focus area re-selection processing (S424) by theAF control unit 304, the AE control unit 302 shifts to the tracking OFFmode so as to temporarily stop the subject tracking processing. In thiscase, the AE control unit 302 can delete the template image and shiftthe tracking mode, as with from the first frame or can change, if a faceis detected in a predetermined number of successive frames (a pluralityof times in succession), the template image to the new face and startthe face tracking mode again.

According to the present embodiment, in an image capture apparatushaving a subject tracking functionality, if no face is detected, thesubject tracking is performed using a template image that was generatedon the basis of a focus area that was selected by the automatic focusdetection device. It is thus possible to track a subject other than aperson. Further, if a face is detected in a predetermined number ofsuccessive frames after the state in which no face was detected, subjecttracking is performed using a template image that was generated on thebasis of the detected face, making it possible, if the same person isdetected in succession, to automatically track the person (the facethereof).

Further, during tracking of a person (face), the template image is notchanged even if a face is detected as long as it is not determined thatreliability of tracking is low in a predetermined number of successiveframes. Therefore, if the face that is being tracked was temporarilyhidden and then reappeared, it is possible to continue to track the sameface, thereby achieving a stable tracking that suppresses a face of thetracking target from being often changed. Further, during tracking of aperson (face), when it is determined that reliability of tracking is lowin a predetermined number of successive frames, and no face is detectedin the predetermined number of successive frames, then the trackingoperation is interrupted. It is thus possible to prevent focusadjustment from being performed on an unintended subject sequentially.Further, during tracking of a person (face), when it is determined thatreliability of tracking is low in a predetermined number of successiveframes, and a face is detected in a predetermined number of successiveframes, then the template image is updated on the basis of the detectedface. This makes it possible, if the tracking target is lost, to changeto tracking of the currently stably detected face automatically andpromptly.

Other Embodiments

Aspects of the present invention can also be realized by a computer of asystem or apparatus (or devices such as a CPU or MPU) that reads out andexecutes a program recorded on a memory device to perform the functionsof the above-described embodiment(s), and by a method, the steps ofwhich are performed by a computer of a system or apparatus by, forexample, reading out and executing a program recorded on a memory deviceto perform the functions of the above-described embodiment(s). For thispurpose, the program is provided to the computer for example via anetwork or from a recording medium of various types serving as thememory device (e.g., computer-readable medium).

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2012-089675, filed on Apr. 10, 2012, which is hereby incorporated byreference herein its entirety.

What is claimed is:
 1. An image capture apparatus comprising: a facedetection unit configured to detect a face that is included in an image;a subject detecting unit configured to detect, from an image obtained byshooting, a area that is similar to a template image; and a control unitconfigured to achieve a functionality of tracking a subject by makingthe subject detecting unit sequentially detect a area similar to atemplate image of the subject area in images that are continuously shot;wherein the control unit is further configured to determine, duringtracking of the subject, reliability of the detection by the subjectdetecting unit and, when the face detected by the face detection unit isset as the template image, not to change the template image until it isdetermined a predetermined plurality of times in succession thatreliability of the detection by the subject detecting unit is low. 2.The image capture apparatus according to claim 1, wherein the controlunit is further configured to change, during tracking of the subject,the template image to another face that has been continuously detectedfor a predetermined plurality of times in succession if it has beencontinuously determined for a predetermined plurality of times insuccession that reliability of the detection by the subject detectingunit using a face, which was previously detected and has been set as thetemplate image, is low.
 3. The image capture apparatus according toclaim 1, wherein the control unit is further configured to temporarilyinterrupt the tracking of the subject using the template image if noface has been continuously detected for a predetermined plurality oftimes in succession and if it has been continuously determined for apredetermined plurality of times in succession that reliability of thedetection by the subject detecting unit using a face, which waspreviously detected and has been set as the template image, is low. 4.The image capture apparatus according to claim 1, further comprising: afocus detection unit configured to detect focus conditions for aplurality of focus areas and to select a focus area to be focused on thebasis of the focus conditions, the control unit is further configured,when the template image is not set, to set the template image on thebasis of the focus area selected by the focus detection unit.
 5. Theimage capture apparatus according to claim 4, wherein the focusdetection unit is further configured, when a subject is detected by thesubject detecting unit, to select a focus area to be focused on thebasis of the location of the detected subject.
 6. The image captureapparatus according to claim 4, wherein the face detection unit isfurther configured to detect the face on the basis of an image having afield of view that is observed through a transmissive type displaydevice for displaying at least the focus area to be focused among theplurality of focus areas.
 7. A method for controlling an image captureapparatus, the image capture apparatus comprising: a face detection unitconfigured to detect a face that is included in an image; and a subjectdetecting unit configured to detect, from an image obtained by shooting,a area that is similar to a template image, the method comprising: acontrolling step of achieving a functionality of tracking a subject bymaking the subject detecting unit sequentially detect a area similar toa template image of the subject area in images that are continuouslyshot, in the controlling step, during tracking of the subject,reliability of the detection by the subject detecting unit beingdetermined, and when the face detected by the face detection unit is setas the template image, the template image being not changed until it isdetermined a predetermined plurality of times in succession thatreliability of the detection by the subject detecting unit is low.